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A new dwindling nonmonotone filter 
method without gradient information 
for solving large-scale systems of 
equations 


F. Arzani and M.R. Peyghami* 


Abstract 


In this paper, we present a new derivative-free spectral residual method 
for solving large-scale systems of equations. Our algorithm is equipped with 
a dwindling multidimensional nonmonotone filter in which whose envelope 
is dwindling as the step-length of line search is decreasing. The proposed 
algorithm is also combined with a relaxed nonmonotone line search technique 
which allows the algorithm to enjoy the nonmonotone property from scratch. 
Under some standard assumptions, the global convergence property of the 
proposed algorithm is established. Numerical results on some test problems 
show the efficiency and effectiveness of the new algorithm in practice. 


Keywords: Dwindling filter technique; Systems of equations; Nonmonotone 
line search; Global convergence. 


1 Introduction 


In this paper, we deal with the following nonlinear systems of equations: 
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F(x) = 0, (1) 


where F’: R” > R” is a continuously differentiable map. We assume that (0) 
is symmetric; that is, its Jacobian is a symmetric matrix. Associated with 
(M), the norm function is defined by 


f(a) = SIF @)P. 


Several methods in the literature have been proposed for solving (fH), such 
as Newton and quasi-Newton methods, Gauss-Newton method, Levenberg— 
Marquardt method [20], trust-region methods, spectral gradient, and residual 
methods [10,24] and conjugate gradient methods [9]. The main factor in 
these methods is how the procedure deals with large-scale settings. The 
spectral gradient and conjugate gradient methods are a class of methods 
that can suitably cope with large-scale systems of equations. The spectral 
gradient method was first introduced in [@ for unconstrained optimization 
and has been successfully extended for solving large-scale nonlinear systems 
of equations; see, for example, [25,29], and the references therein. 


The so-called nonmonotone line search techniques for unconstrained op- 
timization was introduced by Grippo, Lampariello, and Lucidi in [1ZJ. In 
their approach, at the point x;, for a given positive integer M, the stepsize 
A € (0, 1] is chosen such that 


f (te + Adz) < f (xe—j) +YAVF (te) de, 


max 
0<j<min{k,M} 


where 7 € (0,1) is a given scalar. An extension and adaption of this technique 
in the framework of a globalization technique for the Barzilai-Browein (BB) 
gradient method has been proposed in [8]. It is well known that the Grippo’s 
nonmonotone technique suffers from some drawbacks; some of them have 
been listed in [I]. In order to overcome these difficulties, Ahookhosh, Amini, 
and Peyghami in [2] introduced a nonmonotone term as follows: 


Re = €rfecry + 1 — €x) fe, (2) 
where 
feck) = m Sk-j; (3) 


= ax 
0<j<min{k,M} 


fr = f(ax), and ex € [émin; €max| C [0,1]. The motivation behind this non- 
monotone term is that the best convergence results are obtained by stronger 
nonmonotone strategy whenever the iterates are far from the minimizer and 
by weaker nonmonotone strategy whenever iterates are close enough to that, 
see, for example, [20]. Although, (2) enjoys the strong and weak nonmono- 
tone properties; it does not allow the iterates to act as a nonmonotone iterate 
in the first iterations. Ataee Tarzanagh, Saeidian, Peyghami, and Mesgarani 
in [6] proposed a nonmonotone term by replacing Ry with (1+ 4,) Rx in the 
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trust region ratio, where w, is determined by 


ab _ J if R, > 0, 
a ie if Ry < 0, 


and {7} is a positive sequence satisfying the following condition: 


inn om (4) 
k=1 


This substitution causes an increase in the function value in the first itera- 
tions. 


Since the Jacobian of () might not available or requires a prohibitive 
amount of storage, derivative-free methods have been widely developed for 
solving large-scale nonlinear systems of equations. For the square systems, 
La Cruz, Martinez, and Raydan in [24] proposed a derivative-free spectral 
residual method, named as DF-SANE, based on a combination of the Li and 
Fukushima’s line search [26] and the Grippo’s nonmonotone technique. In 
their method, the step-length a; is determined by one and only one of the 
following two conditions: 


f(t —ondn) < felk) + me — you!||F (rx)II7, 
f(tr +ondn) < felk) + me — you||F (rx) II7, 


where dy := o,F'(x,) and ox is the BB spectral coefficient, which is computed 
by [dj 


T 
5S; Sk 


On = 
Tt ? 
Yi, Sk 


where s_ = Ue41 — ep and yp = F(xg41) — F (ag). 


Several variants of DFSANE algorithm for the square systems have been 
proposed and applied on some specific systems of equations; see, for example, 
[(21H23]. Cheng and Li in [LI] introduced another derivative-free spectral 
residual method for the square systems, named as NDFSANE, in which the 
nonmonotone term f¢x) is replaced by that of proposed in [BO]. Later, the 
performance of DFSANE algorithm for the square systems is improved in 
SDFSANE algorithm [10] by using an approximation of the steepest descent 
direction. 


Recently, La Cruz in [22] presented a residual spectral algorithm for solv- 
ing monotone equations on the Hilbert space which is an extension of the 
SANE and DFSANE algorithms for systems of equations. This algorithm 
uses in a systematic way the residual d = F(x) as a search direction, com- 
bined with a suitable step-length and a nonmonotone line search globalization 
strategy. The main objective of this algorithm is to guarantee the conver- 
gence without any necessity to the differentiability of F’. 
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The concept of filter for constrained optimization problems was first in- 
troduced by Fletcher and Leyffer in [15] in order to avoid the difficulty of 
adjusting penalty parameter in penalty methods. Accepting/rejecting of a 
trial step in the filter methods is inspired from the same idea in multiobjec- 
tive optimization. Indeed, a trial point is accepted by the filter if it is not 
dominated by any other points in the filter. In this situation, the filter is 
updated by removing all the points that are dominated by the new point; 
see, for example, [15,116,27], and the references therein. The fixed envelope 
of a multidimensional filter keeps an acceptable trial point far from iterates 
in the filter. However, the trial point with a tiny step-length may get close 
to the current iterate and be rejected by the filter due to its fixed envelope. 
Due to this fact, Chen and Sun in [R] proposed a dwindling multidimensional 
filter line search method for unconstrained optimization. Their idea is to 
incorporate step-length in the framework of multidimensional filter, and let 
the envelope of the filter become thin as the step-length approaches zero. 
Recently, Arzani and Peyghami in [4] proposed a dwindling filter technique 
for solving positive definite generalized eigenvalue problem. 

Our aim in this paper is to introduce a new dwindling nonmonotone filter 
DFSANE method for solving large-scale nonlinear systems of equations. Our 
approach is a new version of DFSANE algorithm which is installed by a filter 
technique. The concept of filter leads to save some points that are eliminated 
by the DFSANE algorithm. The accumulated points in the filter helps the 
algorithm to reach the optimal point as quickly as possible. Moreover, our 
approach uses some relaxations in its structure and is equipped with the non- 
monotone term as proposed in [6]. Besides, by using dwindling technique, we 
have more flexibility for the acceptance of the trial step. Under some stan- 
dard and suitable assumptions, the global convergence property of the new 
proposed algorithm is constructed. Numerical results on some test problems 
validate that the proposed algorithm is practically efficient and robust and 
it has some priorities to that provided in [BI]. 

The rest of the paper is organized as follows. Section 2 is devoted to 
describe the new algorithm in details. In Section 3, we establish the global 
convergence property of the new proposed algorithm under some suitable 
assumptions. Some preliminary numerical results of applying the new algo- 
rithm on some test problems are given in Section 4. Finally, we end up the 
paper by some concluding remarks in Section 5. 


2 The new algorithm 


In this section, we propose a new dwindling nonmonotone filter DFSANE 
algorithm for solving large-scale nonlinear systems of equations. To do so, 
we first recall the filter that is used in this paper and then describe the new 
algorithm. 
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Recently, Fatemi and Mahdavi-Amiri in [13] introduced a new filter that 
controls the size of the filter. In their proposed filter, a tentative point x, is 
acceptable with respect to x; if there exists 7 € {1,...,n} such that 


[95 (ae) |" + 82 IIg(re I" < log(@)l + 1 IIg(e)II" (5) 


where g(x) = Vf (x), 41, and pg are positive constants and 6; and 62 satisfy 
the following relation: 


1 
0< 0, < 02 < —. 6 
1<&< (6) 
Therefore, a point x, is accepted to the filter F if it is accepted with respect 
to every x; with g(a,) € F. In the case of acceptance of xz, g(x) is added 
to the filter and any g(a,) € F with the following property is removed: 


[95 (ae) |" + 82 lowe)" < log(@) + Or Iga)" VF € {1,--. mF. 


In our dwindling filter, the parameters 0; and 02, satisfying (@), are consid- 
ered as a function of the step-length. Since the step-length a in line search 
techniques is normally less than or equal to one, in our setting, we use the 
following parameters in the filter (@): 


6, = ¢(a)O, and 62 = ¢(a)b2, 


where 6; and 62 satisfy (@) and ¢: [0,1] + R* is a monotonically increasing 
continuous function with the following properties: 


a->0t @ 
g(a) =0 if and only if a=0 (7) 


such that (6) holds for 6; and 62. 

Now, let us briefly describe one iteration of our new algorithm. Given zx, 
if the stopping criteria do not hold, the trial step d, = —o,F, is computed 
where ox is the so called spectral coefficient. Assuming a, = a_ = 1, the 
trial point a = ¢,+a+4d,z and py = Tk a_d;, are introduced and with 
the priority of coe 1 are verified for acceptance to the filter. In case of rejecting 
both trial points by the filter, these points with the same priority are checked 
for acceptance by a nonmonotone line search conditions. If the trial points are 
rejected by the filter and line search conditions, then a backtracking scheme 
is done on a; and a_ and the above mentioned procedure is repeated until 
one of the trial points is accepted by the filter or nonmonotone line search 
conditions. 

The structure of the new proposed algorithm is outlined in Algorithm 1. 
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Algorithm 1. : A new dwindling derivative free filter algorithm 


Step 0. Given xp € R”, integer M > 0,0<7< 1,€>0,0 < onin < 
Omax < ©, 0 < 0, < 02 < Tu 0 < Tmin < Tmax < 1, and a monoton- 


ically increasing continuous function @ satisfying (Q); set Fo = © and 
k:=0. 


Step 1. /f ||F(xx)|| <€, then stop. 


Step 2. Choose the spectral coefficient o, such that \ox| € [Omin, Omax], and 
compute fox) by using (B). Set dy, = —o,F (rz), 24 =1 anda_ =1. 


Step 3. Compute ey =r, +a4dz, t,4, = 2 —a_dg, and Ry, by (). 
If Aa is accepted by the filter, then set p41 = ig and add x%41 to 
the filter F;,. Update the filter to form Frii. Setk:=k+1, an = a4, 
and goto Step 1. 
Else if tj, 18 accepted by the filter, then set te41 = Ty ,, and add 
tp41 to the filter F,. Update the filter to form Fria. Setk:=k+1, 
Qp = a_, and goto Step 1. 


Else if 

Phy.) < (1+ Ga) Re — 704? f(x); (8) 
then set tp41 = as k:=k+1, a, =a4, and goto Step 1. 
Else if 

Sf (tha) S At dn) Re — yo-? f (ax), (9) 


then set Fp41 = 241, k:=k+1, ax =a_, and goto Step 1. 


Step 4. Choose a+ € [Tmin@+,Tmaxa+] and a— © [Tmin@—,Tmaxa_], and 
goto Step 3. 


Remark 1. Since 7, > 0, Ry, > 0, and R;, > f(x,), due to Lemma 2.2 in [2], 
it is easily seen that after finite number of reductions of a, or a_ in Step 4 
of Algorithm [], one of the conditions (8) and (@) is necessarily satisfied. This 
shows that the proposed algorithm is well-defined. 


A new dwindling nonmonotone filter method without gradient ... 25 


3 Convergence analysis 


In this section, our goal is to analyze the global convergence property of 
Algorithm [. For this purpose, let 


Pl= {klap E Fy}, 


and let 
P2 = {k|x,, satisfies (8) or (Q)}. 


It is easily seen that P1M P2 = ©. Before going further, we first consider 
the following assumptions on the problem ({): 

Al. F is a continuously differentiable function map over R”. 

A2. For a given 7, as in (@), the level set L° = {x € R"|f(x) < e"f(ao)} 
is closed and bounded. 

A3. The sequence {z,}, generated by Algorithm [], is contained in a 
bounded and closed set 2 Cc R”. Let {x,} be the sequence generated by 
Algorithm [I Then, Algorithm [] either stops at a certain iteration ky with 
||F(xx,)|| = 0, or generates an infinite sequence. In the latter case, it is 
shown that there exists a subsequence of {;,} which converges to a stationary 
point of F. Therefore, from now on, we assume that |P1U P2| = oo. Two 
possibilities are recognized for the cardinality of Pl and P2, |P1| = co or 
|P2| = co. 


3.1 Analysis of the case |P1| = co 


Consider the filter F, and its acceptance criterion, which is given by (8). 
Due to Lemma 4 in [13], the size of F; is finite. Although, 7, contains fi- 
nite number of elements; it may happen that an infinite number of iteration 
points is accepted by the filter. The following lemma states the behaviour of 
the algorithm in this case. 


Lemma 1. Let Assumptions A1—A3 hold, and let |P1| = 00. Then, 


dim, p, |/Fesll =0. (10) 


Proof. The proof is similar to the proof of Lemma 5.4 in [[4] by setting 
0(x) = F(a) and using the fact that based on the definition of ¢(@) and the 
initial values of ay = a_ = 1 in Step 2, the condition 0 < 0, < 62 < Ta 


implies that 0 < 6, x 6. < Sa 
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From Lemma 1, for the case |P1| = oo, it is concluded that there exists 
a limit point of subsequence {xz}pep1 which is a stationary point of f; that 
is, 


lim _ inf ||F,|| =0. (11) 
k-o00, kEP1 


3.2 Analysis of the case |P2| = co 


First of all, without loss of generality, we may assume that |P2| = oo and that 
|P1| is finite, as for the case |P1| = co (ZI) holds. Under this assumption, 
we show that there exists a limit point of subsequence {x,},ep2 satisfying 
the first-order necessary condition; that is, 
lim __ inf ||F,|| = 0. 
k—- 00, ke P2 
Since |P1| is finite and |P2| = oo, there exists a positive integer Mp such 
that for every k > Mo, we have k € P2. Without loss of generality and in 
order to establish simple analysis, in what follows, we assume that Mp = 0, 


and therefore 
P2=40,1;2550:}. (12) 


To construct convergence property of Algorithm [] in this case, we first state 
some technical lemmas. 


Lemma 2. Suppose that {a~}kep2 is generated by Algorithm fl. Then, we 
have 


k 
fear S lfol [[ G+ vi) — oe, 
i=0 
where wy = yar? f (Zz). 


Proof. The proof is similar to the proof of Lemma 3 in [5], and therefore we 
omit it here. 


The following lemma shows that all the iteration points that are gener- 
ated by Algorithm [] are contained in the level set L°. Using Lemma 2, its 
proof is the same as the proof of Lemma 4 in [5] and Lemma 3.4 in [28]. 


Lemma 3. For the sequence {xx}nep2, generated by Algorithm f, we have 
{xx} C L°. 


Remark 2. Let {%%},ep2 be generated by Algorithm I, and let |P2| = oo. 
Then, due to the (12), for a given positive integer M, one can rewrite P2 as 
below: 
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P2={LM+r|LeNu{0},0<r<M—1I}. 


Lemma 4. Assume that {xx~}nep2 is an infinite sequence, generated by 
Algorithm 1 and that M is the constant as given by (@). Then, for every 
k € P2, there exist a non-negative integer L andO <r < M—1 such that 
k=LM-+r and 


LM+r L 


Stevi = fumM+r+41 < | fol II (1 +i) _ So ws(i): L= 0,1, 252256 


1=0 1=0 


where Ws) = w; and w; is defined as in Lemma 2. 


min 
iM<j<(i+1)M-1 


Proof. The proof is similar to the proof of Lemma 6 in [5]. 


Lemma 5. Suppose that Assumptions Al and A2 hold and that {xz}nepa 
is the infinite sequence generated by Algorithm. Then, one has 


lim wes) = 9, 


where w.(;) ts defined as in Lemma 4. 


Proof. Using Lemmas 3 and 4, for 1 <r < M and LM+reT, we have 


L LM+r-1 
S > ws(iy < | fol II (1+ i) — fom+r < e"| fol — frmtr- 
i=0 i=0 


By taking limit from both side of this inequality, as L — oo, and using 
Assumption A1, we conclude that pyran W.(4) < 00, which implies that 
—00 


7 


This completes the proof of the lemma. 


From now on, we define K = {s(i)|i = 1,2,...}. The following theorem 
establishes the global convergence of Algorithm [IL 


Theorem 1. Let Assumptions A1-A3 hold, and let {x,} be the sequence 
generated by Algorithm fl. Then, for every limit point x* of {xp}rex, one 
has 

(J(a*)? F(x*), F(a*)) = 0, 
where (.,.) stands for the inner product of two vectors in R”. 


Proof. From Lemma 5, we have 


Ws(k) = ad mi 705 |F(23)|/? = 0, 


lim — im n 
kEK, k00 ke K, k—-00 kM<j<(k+1)M-1 
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which implies that 
li 2\|F 2 =) 14 
pe Be yoo CHP eI (14) 
using the fact that 0 < y < 1. Since a* is a limit point of {a,}x«, there exists 
an infinite index set K, C K such that limpex,, p00 Te = X*. Besides, (4) 
implies that 
li 2\|F ei. 15 
cals, OFF (onl (15) 
To proceed, we consider the following two possible cases: 
Case 1: lime Ky Ak x 0. 
In this case, there exist an infinite sequence of indices Ky C K, and a positive 
constant c such that a, > c > 0 for all k € Ky. Therefore, (14) implies that 


é 2 
dim |lF(ex)|I" = 0. (16) 


Now, (5) along with the continuity of F imply F(2*) = 0. 
Case2: lime Ky Ak = 0. 
In this case, there exists kg € Ky such that a, < 1, for all kg < k € Ky. 
Therefore, for infinitely many iterations, the line search is not immediately 
successful and a; and a_ should be updated at least once based on Step 4 
of Algorithm 1. Suppose that, in step k € Ky, a, and a_ are adapted my, 
times in the line search procedure in Steps 3-4. Let aj and a; be the values 
of at and a~, respectively, in the last unsuccessful line search process; that 
is, inequalities (Q) and (Q) are violated. Now, form Step 4 of Algorithm [], 
for all kg < k € Ky, we have 

Aa, = goes 
This inequality together with limgex, a, = 0 and the fact that Trin < 1 
imply that 


lim mp =o. 
kek, 


On the other hand, Step 4 of Algorithm M implies that af < 7™*>1 and 


a, <7ik-', Therefore, using the fact that Tmax < 1, we deduce that 


lim at = lima, =0. 
keK1 keK1 

Now, from the definition of at and a, , for all k > ko, k € Ki, we have 
f(a, — of oF (ae)) > (1+ de) Re — Vot)’ f(r), (17) 
f (oe + a, onF (2g)) > (1+ de) Re — Yay)? F (ae). (18) 


Since yx > 0 and Ry > 0, then (1+ vx) Re > Ry = 0. Besides, from Lemma 
2.2 in [2], one has Ry, > f(x,). Thus, (72) implies that 


f (ax — af on F (ee)) — f(x) > —y(agt)?F (ax). 
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From Lemma 3 and Assumptions A1-A2, {f(x,)} is a bounded above se- 
quence. Thus 


f (ax — at onF (xx)) — f (xe) > —y(at)?e"f (ao), 
which implies that 
F(x — of onF(xx))\|” — ||F(@e II? > —y(at Pe" F (20), 
and therefore, 


Flex — of onF os)" — Full” 


+ 
Ay 


ya; e" f (xo). 


Now, by Mean Value theorem, there exists € € [0,1] such that 
on(g(an — Exax F(xx)), -—F(an))) > —e"f (wo) y0% - (19) 


Due to Algorithm [I], two possible cases may happen for oz, it might be 
positive or negative for infinitely many indices. If a; > 0 for infinitely many 
indices k, then (1) implies that 


+ 
VOR 


Omin 


+ 
(g(x — Geog Fe), F(e))) Se" F(0)-* < e"f(vo) (20) 


Using an analogous approach, from (18), we conclude that 


(glee + & a Fee), F(e))) 2 ef (0) > —e"f(eo) 2, (21) 


Omin 


for some € € [0,1]. 
Now, asa; > 0, a, — 0, and ||o,F'(xx)|| is bounded, taking limit from both 
sides in (20) and (21) leads to the following equality: 


(J(2*)? F(a*), F(a*)) = 0. (22) 


For the case in which ox, < 0, for infinitely many indices, proceeding in a 
similar approach, we can deduce (22) as well. 


The following corollary is an immediate consequence of Theorem 1. 
Corollary 1. Assume that «* is a limit point of the sequence {xp}keK 
generated by Algorithm fl and that (J(x*)v,v) £0 for all0 Au € R”. Then, 
F(a*) =0. 


Lemma 6. Fork €N, let 
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Ly := f(teemy) = fecem), 


where (kM) € {((k-1)M+1,...,kM} and fej) is defined as in (B). Then, 
we have 


Co 
Desi = F(e(e+1)m)) < [t+ s nt [I] ni] Le. 
i=kM — i=kM 


Proof. Using Step 3 of Algorithm 1, we have 


f(tem4i1) < A +dem) Rem < (14+ vem) Le 

f (tem+2) = (1 + demi) [ex max{ Le, f(Ceargi)} + (1 — ex) f (tem 41)] 
(1 
= [1 


+ Wem4i)(1 + vem) Le 
+ dem + Vemai + Vem Vem 4i)Le, 


and so on. Therefore, by an inductive argument, 


kM+j kM+j 
f(@em+j) < 1+ S- m+ II s| Ly. 
i=kM i=kM 


Since (kh +1) € {kM +1,...,(K +1)M}, then, we have 


1+ > m+ Il " Ly. 


i=kM i=kM 


Les = f(y(e-41)m)) 


This completes the proof of the lemma. 


Let us define Ky, as below: 
Ky, = {€(1), €(2),...}. 
One can easily observe that 
gj +1) < €y)+2M —-1, § SL Dien (23) 


The following theorem states that in case of existing a limit point for the 
sequence generated by Algorithm [J all the limit points are the solutions of 
the nonlinear system ({). 


Theorem 2. Let x* be a limit point of the sequence {xz}, generated by 
Algorithm fl, such that F(a*) =0. Then, we have 


lim F(a) = 0. 
k-00 


A new dwindling nonmonotone filter method without gradient ... 31 
Proof. Let Ky be an infinite subset of N such that 


lim  @2,=a”*. 
ke Ky1,k-400 


This relation together with Assumption Al imply that 


F (xz) = 0. (24) 


lim 
k€K1, k—00 


Now, since p41 = ©, £ apo, F (xz) and laxox| < Cmax, for all k € N, using 
(24), one can deduce that 
lim |le@x41 — re|| = 0, 
ke Ky, kk 00 
and therefore, 
lim = p44 = 2". 
kEK1, k-400 


Proceeding by induction, one can easily show that for the fixed j € {0,1,..., 
2M — 1}, we have 
li ,= 2", 2 
Pe cia a a en) 
Now, using (23), for a given k € Kj, there exists y(k) € {0,1,...,2M — 1} 
such that k + u(k) € K,. Since K, is infinite, there exists 7 € {0,1,..., 
2M — 1} such that, u(k) =7 for infinitely many k € K,. Assume that 


Ky ={k + p(k)|k © Ky and p(k) = 7}. 


It is easily seen that Ky C K+, and by (24) 


lim tea. = 2". 
k€ Ko, k00 
Thus 
lim F(a,) =0. 
k€ Ky, k00 


Now, as Kz C K,, there exists an infinite subsequence {2x,9(;)}je such that 


lim #4) =x" and ne F(%a)) = _ L; =0. (26) 
Let us write J = {j1,j2,j3,-.-$, where j1 < jo < jg <-++-. Then, (20) 
implies that 
lim L;, =0. (27) 
woo 


Now, from Lemma 6, we have 


co co 
Ips |1+ So m+ [[ mil, VoeN p> i. 
t=Mj; t=Mj; 


32 F. Arzani and M.R. Peyghami 


Thus, 


oo co 
supLp < |1+ S- m+ II me | Lj,- (28) 
Pri t=Mji t=M ji 


On the other hand, from (4), we have 


co Co 
l = 
jim | D7 m+ TL m] =0 
t=M ji t=M ji 


Therefore, using (27) and (28), we conclude that 


lim supL, = 0, 


UEOO pe Gs 
which in turn implies that 


lim L; = 0. 


j- co 


Now, using the definition of L;, we have 


Ate 2 
jim f(x) = Jim 5 |lF(ae) I" = 0. 


This completes the proof of the theorem. 


4 Numerical results 


In this section, our aim is to investigate the practical performance of Algo- 
rithm 0, denoted by DF-DFSANE, along with the following algorithms on 
some test problems: 


e DFSANE: Algorithm 1 in [24]. 


e NF-DFSANE: Algorithm Min this paper in which the concept of filter 
has been removed. 


All the considered algorithms are implemented in MATLAB 7.10.0 (R2010a) 
environment on a PC with CPU Intel 2.33 GHz Quad, 4GB RAM memory, 
and double precision format. Test problems are mostly taken from [8,9], 
and the problem dimensions vary from 2 to 10000. 

All the considered algorithms are being stopped whenever || F(x,) ||< 
10~-°. Moreover, we declare that an algorithm is failed whenever the number 
of iterations and the number of function evaluations exceed 10000 and 50000, 
respectively. We have also utilized the advantages of the performance profile 
of Dolan and Moré [TQ] in order to properly compare the considered algo- 
rithms. The following parameters are considered in the relevant algorithms: 
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€= 10°, 1 = 0.25, ug = 0.75, omin = 10°”, Omax = 107°, 


1 
M = 20,7 = 10-4, tmin = 0.1, Tmax = 0.5, Me = ————s, F > 0. 
77 37 T, Nk (1+k)? tale 


Moreover, we utilized $(a) = a? in the dwindling technique. It is worth 
mentioning that the parameters are chosen from a set of values in which rea- 
sonably better results are achieved while performing an algorithm. Numerical 
results are given in Table 1. In this table, Problem, n, n;, and nz represent 
the problem name, the problem size, the number of iterations, and the num- 
ber of function evaluations, respectively. Based on the results of Table 1, we 
have plotted the performance profile of the considered algorithms in Figures 
1 and 2 in terms of n; and ny, respectively. 


At a glance to Figure 1, one can easily find out that DF-DFSANE al- 
gorithm solves roughly 93% of the test problems successfully while this per- 
centage for NF-DFSANE and DFSANE algorithms is below 90%. Moreover, 
DF-DFSANE algorithm solves about 62% of problems at the lowest value of 
n; while this percentage for the NF-DFSANE and DFSANE algorithms are 
40% and 10%, respectively. 


Figure 2 reveals that when the DF-DFSANE, NF-DFSANE, and DF- 
SANE algorithms are applied on the test problems; the DF-DFSANE al- 
gorithm solves roughly 49% of the test problems in the lowest value of nf 
while this number for the NF-DFSANE and DFSANE algorithms are 38% 
and 10%, respectively. Moreover, DF-DFSANE algorithm solves about 90% 
of the test problems without any failure. It has to be noticed that in the 
cases that DF-DFSANE is not the best algorithm; its performance index is 
roughly close to the performance index of the best algorithm. 


vii! DFSANE 
ait === NF-DFSANE| 
—— DF-DFSANE 


0 L 1 1 Ll Ll 
1 2 2 4 5 6 7 


Figure 1: Performance profile of the considered algorithms in terms of n,. 
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Figure 2: Performance profile of the considered algorithms in terms of nf. 


5 Conclusion 


In this paper, a new dwindling nonmonotone filter DFSANE method for solv- 
ing large-scale nonlinear systems of equations is proposed. In our approach, 
a filter is set up on the so-called DFSANE algorithm which has been devel- 
oped in [24]. Indeed, the concept of filter helps us to save some appropriate 
points that are eliminated by the DFSANE algorithm. Moreover, the accu- 
mulated points in the filter cause the algorithm to reach the optimal point 
as quickly as possible. The proposed approach also uses some relaxations in 
its structure and is equipped with the nonmonotone term as proposed in [6]. 
Furthermore, we have more flexibility for the acceptance of the trial step by 
using a dwindling technique. Under some standard assumptions, the global 
convergence property is established. Numerical results on some test problems 
confirm that our proposed algorithm is practically efficient, robust, and has 
some priorities to that provided in [BI]. 
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Table 1: Test problems and numerical results. 
DFSANE NF-DFSANE DF-DFSANE 

Problem n ni/n¢ ni/nz ni/n¢ 
Extended Beale [3] 1000 38/52 33/46 29/39 
Extended Beale [3] 5000 53/79 47/57 38/52 
Extended Beale [3] 10000 62/90 48/72 38/64 
Extended penalty [B] 1000 17/39 13/35 10/24 
Extended penalty [J 5000 21/49 17/45 21/47 
Extended penalty [B] 10000 23/49 21/51 22/50 
Extended Three Exponential [Bj 1000 18/32 16/24 15/19 
Extended Three Exponential [Bj 5000 29/43 18/32 14/30 
Extended Three Exponential [Bj 10000 30/62 20/34 20/32 
Generalized Tridiagonal-2 [B] 1000 58/68 58/72 58/72 
Generalized Tridiagonal-2 [B] 5000 Failed 193/296 126/258 
Generalized Tridiagonal-2 [B] 10000 Failed 789/3299 440/1083 
Extended PSC1 Function [BJ 1000 18/26 14/21 14/20 
Extended PSC1 Function [BJ 5000 22/42 18/26 17/23 
Extended PSC1 Function Bj 10000 36/42 22/34 17/23 
Extended Block Diagonal BD1 Bj 1000 17/23 17/23 15/23 
Extended Block Diagonal BD1 [BJ 5000 19/25 15/23 20/24 
Extended Block Diagonal BD1 [BJ] 10000 99/131 114/213 105/285 
DQDRTIC (CUTE) [3 1000 64/116 52/74 50/70 
DQDRTIC (CUTE) [BI 5000 —-89/209 60/90 35/61 
DQDRTIC (CUTE) [BI 10000 126/301 44/66 33/47 
LIARWHD function [BJ 1000 1/1 1/ 1/1 
LIARWED function [BJ 5000 18/90 18/90 16/74 
LIARWED function [BJ 10000 ~—- 87/233 111/401 105/341 
Extended DENSCHNF [Bj 1000 1/1 1/ 1/1 
Extended DENSCHNF [3] 5000 25/41 21/49 21/49 
Extended DENSCHNF [3] 10000 «52/76 25/41 22/40 
Generalized Quartic [Bj 1000 17/23 10/12 10/10 
Generalized Quartic [Bj 5000 20/28 10/12 2/4 
Generalized Quartic [Bj 10000 50/98 21/29 12/16 
Diagonal 8 [Bj 1000 8/9 6/7 5/6 
Diagonal 8 [BI 5000 20/37 16/17 7/9 
Diagonal 8 [Bi 10000 ~—- 38/95 5/38 7/9 
Full Hessian FH3 [3] 1000 4/24 4/24 4/22 
Full Hessian FH3 [B] 5000 5/31 4/30 4/28 
Full Hessian FH3 [Bl] 10000 Failed Failed Failed 
SINCOS [BI 1000 18/26 14/21 11/16 
SINCOS [BI 5000 27/33 18/26 17/23 
SINCOS [Bj 10000 = 36/42 30/36 17/23 
HIMMELH (CUTE) [5 1000 1/1 1/1 1/1 
HIMMELH (CUTE) [pi 5000 14/18 14/18 8/10 
HIMMELH (CUTE) [3] 10000 Failed Failed Failed 
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Table 1: Test problems and numerical results (Continued). 


DFSANE NF-DFSANE = DF-DFSANE 


Problem n nj /nF n;/nF nj /nF 
Power [3] 1000 75/91 74/90 54/62 
Power [3] 5000 152/176 123/142 120/134 
Power [J 10000 224/345 210/311 129/135 
FLETCHCR function (CUTE) [3] 1000 750/1404 712/1201 425/689 
FLETCHCR function (CUTE) [Bj 5000 Failed 1012/4865 994/4752 
FLETCHCR function (CUTE) [BJ 10000 Failed Failed Failed 
Problem 2 [19] 2 46/49 25/31 2/6 
problem 4 [19] 5 1/2 1/2 1/2 
Problem 6 [9] 2 3/7 4/15 3/6 
Problem 7 [19] 5 268/271 62/74 33/42 
Problem 8 [C9 10 15/18 19/25 14/16 
Problem 26 [TU] 5 149/157 21/21 21/21 
Problem 27 [TJ] 2 1/1 1/1 1/1 
Problem 39 [3] 5 1383/2100 24/28 20/34 
Problem 40 [Tj 5 Failed Failed Failed 
Problem 42 [Tj 2 1/1 1/1 1/1 
Problem 46 [3] 5 Failed Failed Failed 
Problem 47 [T3f 0 324/841 260/642 222/423 
Problem 48 [Ul 5 423/1108 112/504 53/71 
Problem 53 [TJ 5 1/1 1/1 1/1 
Problem 56 [J] 0 22/24 3/5 3/3 
Problem 61 [Jj 0 394/5784 357/5650 357/5650 
Problem 63 [9] 5 22/26 47/66 21/38 
Problem 77 [3] 0 149/219 51/126 17/24 
Problem 78 [TJ] 2 Failed Failed Failed 
Problem 79 [TU] 5 970/4130 952/4025 844/3441 
Problem 81 [3] 5 Failed Failed Failed 
Problem 81 [9] 0 Failed 843/4344 432/3456 
problem 111 [19] 5 344/1815 310/1012 72/770 
problem 111 [19] 0 Failed 572/4567 424/3245 
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